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CHAPTER IV 

CRITERIA FOR COUNSELOR PERFORMANCE 



As we Indicated earlier, the need and value of research on the criteria 
for rehabilitation counselor performance is substantial; the development of 
selection, evaluation and training procedures which are research-based can 
only move ahead on a sound, basis after some criteria are set. The value of 
research in these areas will, to a significant degree, hinge upon the type 
and quality of the criteria used. This chapter reports on that phase of our 
research which examined the relationships among rehabilitation counselor per- 
formance criteria currently being used or readily available to state voca- 
tional rehabilitation agencies. Either simple Inspection or cluster analysis 
will show the relationships among these criteria, and will suggest which com- 
bination of criteria will account for most of the variance associated with 
the nine variables studied. 



Procedures of the Study 

Sample ; The 143 counselors studied here came from the following six 
states: Connecticut, Iowa, Minnesota, Missouri, North Carolina and Oklahoma. 

All the agencies selected were middle-sized, general vocational rehabilita- 
tion agencies. As can be seen, some geographic representation was obtained, 
but both accessibility and willingness to cooperate in the study were consi- 
derations in selecting the states. To be included in the study, a counselor 
had to have been employed at least one full year during the three-year period 
studied so that closure and caseload data would be available. Since Co- 
Worker Ratings required at least two co~worker ratings to be an effective in- 
struments, we also limited our sample to counselors who worked in a co-worker 
relationship with two other counselors and a supervisor. 
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Instruraents ! The following performance^, measures and rating procedures 
were used: 

1. The Co-Worker Rating Blank : This blank (Appendix C) Incorporates six 

evaluation dimensions and an overall judgment. It has an eight-step 
scale, and was used by both co<-workers and supervisors to assess the 
counselor performance against what they think Is the Ideal for each 
statement. Only ratings on overall performance were used. 

2. Supervisor Ratings : Supervisors used the same blank as the co-workers 

In making their ratings. 

3. Efficiency Ratings : This label Includes the rating schemes presently 

used In four of the six state agencies and the efficiency ratings made 
by at least two administrators In the two other states which, at the 
time of the study, did not have a systematic, quantifiable procedure 
for evaluating counselors. 

4. Job Satisfaction Inventory : A Job Satisfaction Inventory (JSl) (Ap- 

pendix D), constructed by Johnson (1955), was modified to obtain more 
variability of response and to eliminate nondiscriminating Items. 
Chapter VI reports our research on It. In this part of our research, 
only the total score on the JSI was used. 

5. Average Caseload : The counselor caseload Is obtained by taking all 

of the people a counselor was working with when the fiscal year be- 
gan plus all of the people accepted by him for rehabilitation ser- 
vices In that year. The Index was secured by averaging caseload size 
for the fiscal years 1960, 1961 and 1962. 

6. Average Number of Closures : The same three-year period (1 July 1959- 

30 June 1962) was used to determine the mean number of closures In 
each category: (1) 12 closures - the number of clients rehabilitated; 

(2) 13 closures - those clients to whom services were given but who 
were not judged rehabilitated; (3) 15 closures - those clients who 
were accepted for services but were Interrupted before substantial 
services were given and/or the client became employable. 

7. Caseload Velocity Index : This criterion measures the rapidity with 

which the counselor develops plans for action with the client or pro- 
vides rehabilitation services. To derive It, random samples of ten 
12 closures and ten 13 closures for the 1961-1962 year were taken for 
each counselor. Each case In the sample was checked to see how long, 
by months. It had previously been In Status 1. A mean length of time 
score was used as the Index since some counselors did not close at 
least ten cases In these categories. Time In Status 1 was taken as a 
best index of case movement since we thought that factors which In- 
fluence movement are most directly under the counselor's control 
during this stage. 




With the exceptlcn of co-worker and supervisor ratings, the above data 
were available from the administrative files of each of the state agencies. 
Where our efficiency rating form needed to be used, a global judgment (almost 
always - rarely effective) was secured by submitting It to the state office 
staff responsible for case service activity. The Co-Worker Ratings and the 
Supervisor Ratings based on the Co-Worker Rating Blank were either adminis- 
tered In large groups Including all counselors In the state or by distribu- 
tion to district offices. Precautions were taken so that the Co-Worker Ratings 
would be confidential and would not be seen by either district or state staff. 

Reliability Data ; In two Instances where efficiency ratings were secured, 
a reliability coefficient of .83 was obtained for the average ratings. Average 
ratings on the Co-Worker Rating Blank yielded a reliability coefficient of .87 
for averages analysed by analysis of variance procedures (Lindquist, 1953). 

The reliability coefficient for supervisor ratings was .56. Since we had each 
supervisor rating different counselors, the reliability of supervisor ratings 
were estimated from what the counselor ratings would be If treated individu- 
ally rather than as mean scores. The JSI yielded a split-half reliability co- 
efficient of .91. 



Results 

After Inspection for linearity by the use of scatter-plots, the eight 
variables of Table 10 were Inter correlated, using Pearsonlan correlation. 

Since one of the measures used In our study, the JSI, Is not a performance 
measure, we are not Including it in Table 10 but will discuss it briefly be- 
low. Tables 11 and 12 present cluster analysis findings. 

General Analysis of Intercorrelations ; From Table 10, we can see that 
there is only a limited degree of association among the criteria studied. Even 
the highest correlation, that between size of caseload and number of 12 clo- 
sures, reflects only 36 per cent common variance. An analysis of the average 
Intercorrelation of each variable with all others also showed that caseload, 
which had the highest average, correlated .28 with the others and thus accounts 
for only eight per cent of the variance. To some extent, our discussion of the 
content analysis will also bring out the large amount of specific variance and 
error (i.e. unreliability) variance associated with each Index. As you can see, 
total JSI scores did not correlate significantly with any of the performance 
criteria. 

Cluster Analysis ; Cluster analysis has an objective similar to factor an- 
alysis - the more parsimonious description of a larger group of variables. Al- 
though it is relatively less precise mathematically, and does not give altogether 
the same Information as factor analysis. It does serve the objective of parsimony 
fairly adequately. In addition, it is simpler to do and understand than factor 
analysis. 

In Table 11, we note that two clusters of three variables each were formed. 
The average intra-cluster correlations were .48 and .33, while the average in- 
tercorrelations of the cluster with all other variables were .13 and .11, respec- 
tively. The first cluster obtained might well be called "case management" since 







THE INTERCORREIATIONS OF NINE MEASURES OF REHABILITATION 
COUNSELOR PERFORMANCE IN STATE DVR'S 
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TABLE 11 



CLUSTER ANALYSIS FINDINGS FOR 
MEASURES OF REHABILITATION COUNSELOR PERFORMANCE 



Groups 



Average Correlation 
Within Group 



Average Correlation 
Between Group 
and Others 



B Weight 



Size of Caseload 
Number of 12 Closures 
Number of 15 Closures 



48 



** 



13 



3.69 



II 

Co-Worker 
Present Rating 
Supervisor 



33 



** 



13 



2.59 



** 



[Decimals have been omitted in correlations. 



Significant at .01 level. 
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TABLE 12 

AVERAGE INTERCORRELATIONS OF 
REHABILITATION COUNSELOR PERFORMANCE CRITERIA 
WITH THE TWO CLUSTERS AND ALL OTHER VARIABLES 



Variables 


Group I (4. 6. 8) 


Group II (1. 3. 5) 


All Others 


Co-Worker Ratings 


* 

CM 

O 

1 


30*** 


13 


Case Velocity 


01 


12 


11 


Present State Rating 




36*** 


1CK7C 

25 


Caseload 


56 


15 


28*** 


Supervisor Rating 


17 


1-,1 

wTKTC 

39 


21^ 


Number of 12 Closures 




A ^ ^ ^ K 
22 


27 


Number of IS Closures 


$ 

o 

CM 


08 


12 


Number of 15 Closures 


44 


13 


24*** 



Decimals have been 
**Signif leant at .05 
***Signif leant at .01 



omitted from correlations. 

level. 

level. 
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It Includes size of caseload, number of 12 closures, and number of 15 closures. 
The second cluster might be called "performance rating" since it includes the 
three rating indices of our study. As we can see, neither case velocity nor 
number of 13 closures clustered with the "case management" cluster, although 
they both correlated to a slight degree with two of the variables of the cluster. 

The average correlation of each variable with the two clusters and with 
all other variables is presented in Table 12. From it, we can see that present 
state rating procedures and supervisor ratings have significant average inter- 
correlations with both clusters, as well as with all other variables. 



Summary 

This chapter reports the relationship of eight performance measures and a j 

measure of job satisfaction when these were applied to 143 rehabilitation conn- / 

selors in six states. A cluster analysis of the intercorrelations showed two 
clusters which included three variables each. The clusters were labelled "case 
management" and "performance rating." Two of the performance variables and the 
job satisfaction scores did not contribute to either cluster. The implications 
of these findings for state vocational rehabilitation agency practice and for 
further research were briefly considered. 
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APPEM)IX C 

V 

CO-WORKER RATING BLANK 



We are asking for your help in a research project which concerns the 
evaluation procedures used towards the work of the rehabilitation counselor. 
This project - under way since November 1959 and supported by VRA - we hope 
will help define some of «*he problems and pertinent dimensions by which such 
an evaluation can be approached. 



One variable we want to study in our project is the perceptions which 
counselors have about the work of fellow counselors. We feel that co-workers 
are perhaps in a good position to know some aspects of that work Since they 
have a chance to Interact and listen to each other - all of which helps them 
form some Idea of how well the other counselor is performing his job. With 
this In mind, we thought that one Important measure of counselor performance 
would be the perceptions of fellow workers. 



The blank should be completed In the following manner: 

1. On the last page of the blank, there Is a list of names sectioned 
Into district offices. In front of each name there Is a number. 

2. From this list, select the counselors which appear In the same sec- 
tion as your name, and put their numbers in the squares at the top 
of page 2 - above the Items to be rated. The Initials, etc., of 
each person may be put below each number in order to help you remem- 
ber who they are. Please rate all counselors in that section. In- 
cluding yourself . (If you are a supervisor, please rate all the 
counselors in the section that has your district office in the title. 
You need not rate yourself.) 

3. On page 2, there are seven Items of a counselor's work on which the 
people you selected should be compared. On the scale for each Item, 
all the numbers you selected should be placed. 

E.g. : 12 t 2.4 : 9 t 8 : 11 : 7.3 : i l5.20 

Is most like Is least like 

4. On each Item, you will compare the behavior, as you see It, of each 
person you selected to the behavior of the best counselors you have 
known - or to what you feel the behavior of the best counselor should 
be. In effect, you are comparing how "like" or similar to your Ideal 
the behavior of the selected counselors corresponds. If It Is "most 
like" your Ideal, then the number should be placed near or at the 
"most like" end, depending on how nearly you feel it is like. If it 
Is "least like," then the other end of the scale should be used, de- 
pending on how strongly you feel It Is "least like" your Ideal. 
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5. Work on one item at a time; put all numbers selected on one scale 
and then move to the next item. 

Your reply will be held in strictest confidence; only the Investigators 
have the coding key which identifies each blank. In our analysis of these 
data, neither individuals, district offices, nor states will be identifiable. 
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CO-WORKER RATING BLANK 



1. In his knowledge of rehabili- 
tation concepts (medical, psy- 
chological, vocational, etc.)* 
the counselor 



Numbers of people to be rated 



Is most like 



Is least like 



2. In collecting sufficient infor- 
mation and using It effectively 
In evaluating a case, the coun- 
selor ; : : : ! ! ! 

Is least like Is most like 



3. In learning from experience and 
showing an attitude of wanting 
to overcome his shortcomings, 
the counselor 

Is most like Is least like 



4. In giving clients appropriate 
Information and In a manner 
which Is meaningful, the coun- 
selor : : : : : : 

Is least like Is most like 



5. In knowledge about and working 
relationships with the major com- 
munity rehabilitation services 
and agencies, the counselor ... ; ; 

Is most like 



Is least like 



6. In creating an atmosphere during 
counseling sessions which makes 
It easier for clients to deal with 
their problems, the counselor .. ; ; 

Is least like 




In the over-all performance of 
his Job, the counselor 



Is most like 




Is most like 



Is least like 
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JOB SATISFACTION INVENTORY 



This inventory is another part of our study into the criteria problem in 
evaluating the work of the rehabilitation counselor. We are sure that most 
would agree that satisfaction with a job is a vital component in the performance 
of that job. This variable we want to Include in our study. 



The following specific statements concern your feelings, beliefs, and atti- 
tudes. There are 70 items in this inventory. For each statement, a five-point 
scale is provided indicating whether you rarely, sometimes, frequently, generally, 
or almost always do, feel, or see others as doing as the statement suggests. Thus, 
for example, you would check the space "R" on the scale if you rarely eneaee in 
the activity described: 

R S F G A 

I worry a lot about my job. x : : : : 



Or, see others as engaging rarely in an activity: 



The policies and problems of the 
people under whom I work are: ade- 
quately explained to me. x : 



To aid you in answering this inventory, the terms rarely, sometimes, fre- 
quently, generally, and almost always have been defined on a percentage basis, 
as follows: 



"R" - 


Rarely: 


means 


from 


0 


to 


15 


per 


cent 


of 


the 


time. ■ 


»S" - 


Sometimes : 


means 


from 


16 


to 


35 


per 


cent 


of 


the 


time. 

5 


"F" - 


Frequently: 


means 


from 


36 


to 


65 


per 


cent 


of 


( 

the 


time. 


"G" - 


Generally: 


means 


from 


66 


to 


85 


per 


cent 


of 


the 


time. 1 


"A" - 


Almost Always: 


means 


from 


86 


to 


100 


per 


cent 


of 


the 


time. ! 

i 


These 


per cent listings have been given at 


the 


top of each 


page in the in- J 



ventory. There are no "right" or "wrong" answers to these statements. Work as 
rapidly as you can without being careless, and do not spend too much time on any 
one statement. 





Answer every question. ] 

rsfga ?^ 

1. If I had a choice, I would choose a job in my I 

present line of work over one In any other line 
of work. 



2. I feel that I have an adequate understanding of 
what is expected of me in my job. 

3. It is necessary for me to do things I dislike 
in order to get promotions. 

^ feel that to me others could make my work 
easier if they cared to do so. 

5. I worry a lot about my daily work. 

6. I feel if I could start over again, at 18, I 
would choose a different line of work. 

7. I feel that people in general respect my job. 

8. There are too many people telling me what to do. 

9. I feel that I can always trust the people under 
whom I work. 

10. My life would seem empty without my work to 
occupy me. 

11. My present job requires me to work too long hours. 

12. I am glad to get back to my job after a vacation. 

13. I feel that I am as efficient as the average per- 
son with whom I wojrk. 

14. My work is too confining to suit me. 

15. I feel I am paid a fair salary for the work I do. 

16. I feel that my work utilizes my full capacities. 
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"R” - Rarely (0% to 15%) 

"S" - Sometimes (16% to 35%) 



"F” - Frequently (36% to 65%) 



III! IWI I ' •• •|•^1t^ 
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”G" - Generally (66% to 85%) 

"A” - Almost Always (86% to 100%) 



Answer every question. 

R S F G A 

17. I feel that I am "in a rut" vocationally. : : : • 

18. I feel that I know where I stand with my 

present employer. .... 



19. I feel that my work has a bad effect on my 
health. 



20. I come home upset, angry or irritable because 
of something that happened at work. 



21. I feel competent and fully able to handle my job. 

22. I feel my work suffers because I have too much 
to do. 



23. I would decline an opportunity to change my 

present job for one of equal pay, security and 
status. 



2A. I think it is possible to attain my vocational 
goals in that portion of life that is still 
ahead of me. 



25. I feel that my family and friends respect my 
vocation. 



26. I feel there is adequate transportation avail- 
able to me in going to and from work, as well 
as in my work when called for. 

27. I think I really wanted to enter my present job 
when I started it. 



2®« I regard my present position as a lifetime career. 

29. I think my present job is in the area of work 
(not necessarily the same job) I wish to remain 
in permanently. 



er|c 




"R" - Rarely (0% to 15%) 

"S" - Sometimes (16% to 35%) 
”F" - Frequently (367* to 657*) 



"G” - Generally (66% to 85%) 

"A" - Almost Always (86% to 100%) 



Answer every question. 



30. I expect my job to give me more satisfaction 
the longer I have It. 



31. I feel I have had definite adequate preparation 
for the job I now hold. 



32. 



33. 



34. 

35. 



36. 



37. 



38. 



41. 



42. 
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I feel I have made real and lasting friends 
among my working associates. 



My position forces me to work with certain 
Individuals whom I dislike. 



I get discouraged In my present job. 



I feel that my job detracts from my status In 
the community where I live. 



I consider my work surroundings to be as pleasant 
as they should be. 



I feel I have eventual retirement security In my 
job. 



I get restless during working hours, and feel 
that the day Is dragging endlessly. 



39. I feel that there should be more people to help 
with the work I am doing. 



40. I like my present job better than any other I 
have ever had. 



My job gives me mere real personal satisfaction 
than the things I do In my spare time. 



I feel my occupation forces me to live In home 
surroundings which are uncomfortable or Inade- 
quate according to my standards. 
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"R" - Rarely (0% to 15%) 

"S" - Sometimes (16% to 35%) 
"F" - Frequently (36% to 65%) 



"G" - Generally (86% to 85%) 

"A" - Almost Always (86% to 100%) 



Answer every question. 



43. 

44. 

45. 

46. 

47. 



48. 



49. 



50. 



51. 



52. 



I wonder whether the people under whom I work 
approve of my work. 



I think my job gets more difficult for me each 
year. 



My present job gets me badly flustered and jit- 
tery. 



The policies and problems of the people under 
whom I work are adequately explained to me. 



I feel that my general Interests and attitudes 
are about the same as those of my fellow workers 
who have similar jobs. 



The method of payment of my earnings causes me 
Inconvenience . 



I feel at ease in the presence of the people 
under whom I work. 



I am so Interested in my work that I talk about 
it a great deal even after working hours. 



I feel I am kept from living as I would like be* 
cause of insufficient income. 



I am satisfied with the degree to which my pre- 
sent job gives me an opportunity to express my 
own ideas. 









53. I find my work so interesting that it is on my 
mind a lot when I am not at wcrk. 

54. 1 feel I have made a success of my job thus far 
in my career. 

55. Ity present job forces me to maintain too fast a 
pace. 
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"R" - Rarely (0% to 15%) 

"S" - Sometimes (16% to 35%) 
"F" - Frequently (36% to 65%) 



"G" - Generally (66% to 85%) 

"A" - Almost Always (86% to 100%) 



Answer every question. 

56. 1 feel that my working associates regard me as 
an equal. 

57. 1 feel that 1 must look outside my work for those 
things that make life worthwhile and interesting. 

58. My income is sufficient to meet my financial 
obligations and support my family. 

59. 1 feel that my associates stimulate me to do 
better work. 

60. I think my job has "smothered" my personality. 

61. My vocational future looks promising to me. 

62. I feel that I am really Interested in my present 
job. 

63. I get along well with the persons with whom I 
work on my present job. 

64. The people under whom I work make available the 
materials. Information and assistance required 
to do my best work. 

65. 1 feel that the people under whom 1 work make 
unfair demands on my free time. 

66. 1 am afraid of losing my job. 

67. I feel that I will become more proficient at my 
work the longer I have it. 

68. Those with whom I work seem unreasonable in their 
dealings with me. 

69. I feel my present job helps me toward the finan- 
cial goals 1 have set for myself. 
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"R" - Rarely (0% to 15%) 

"S" - Sometimes (167p to 35%) 

"F" - Frequently (36% to 65%) 

Answer every question. 

70. The people under whom I work are desirous of 
and willing to make Improvements in my working 
conditions. 



"G*' - Generally (66% to 85%) 

"A" - Almost Always (86% to 100%) 



You may use the rest of this sheet for any comments you would like 
concerning this inventory. Thank you for your help. 



to make 
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